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ABSTRACT 



Using multilevel modeling to analyze survey data from 1,532 
alumni from a large research university, this study proposes to examine the 
impact that departments have on student satisfaction. Controlling for 
individual characteristics, researchers found that characteristics of 
departments such as size, faculty contact with students, research emphasis, 
and proportion of female undergraduates had a significant impact on 
satisfaction with education in the major and the perceived impact that 
college had on skill development. Grade point average (GPA) was a significant 
predictor of satisfaction with education in the major, but there appears to 
be no relationship between GPA and the perceived impact of college on skill 
development. Findings inform policy and practice in two areas. First, they 
suggest that individual attributes matter, and second, the study provides 
evidence of the impact of organizational attributes on student outcomes.. 
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How do academic departments impact student satisfaction? 
Understanding the contextual effects of departments 



Abstract 

Using multilevel modeling to analyze survey data from over 1300 alumni from a large 
research university, this study proposes to examine the impact that departments have on student 
satisfaction. Controlling for individual characteristics, we found that characteristics of 
departments such as size, faculty contact with students, research emphasis, and proportion of 
female undergraduates had a significant impact on satisfaction with education in the major and 
the perceived impact that college had on skill development. 
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How do academic departments impact student satisfaction? 

Understanding the contextual effects of departments 

For decades, scholars and academic administrators have examined the impact that college 
has on students and student satisfaction. Seminal works by Feldman and Newcomb (1969) and 
Pascarella and Terenzini (1991) explore the relationship between students’ college experiences 
and learning, development, and satisfaction. Scholars have long understood the impact of 
subunits within colleges and universities on students and have concluded that they often produce 
quite different influences on student development (Baird, 1988; Chickering, 1969). More 
specifically, several authors have noted the impact that departmental culture and climate have on 
student learning and satisfaction (Cameron & Ettington, 1988; Hartnett & Centra, 1977; 
Volkwein, 1994). 

What do we know about the impact that different departments have on student outcomes? 
In fact we know very little. We do know that academic disciplines vary in their views of 
application of practical problems, cognitive processes, concern with life systems, beliefs about 
collaboration, time commitments and scholarly output (Becher, 1987; Biglan, 1973a; Biglan, 
1973b). Others suggest marked differences between departmental goals (Smart & Elton, 1975). 
Still others indicate differences in course planning and delivery (Stark, Lowther, Bentley, & 
Martens, 1990). While we know a great deal about the differences between departments, very 
little research has been done to study the impact that different departments have on student 
outcomes. 

For researchers, studying the effects of departments poses a dilemma known as the unit 
of analysis problem. In the 1980s, elementary and secondary level researchers began to address 
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school effects when studying students (Burstein, 1980; Burstein & Miller, 1981). They 
recognized that in educational research they are faced with a unit of analysis problem that has 
plagued educational researchers for decades. In the past, researchers were faced with three when 
examining group effects. They could build statistical models that examined data at the group 
level or organizational level and neglect differences in individuals; they could examine data at 
the individual level and ignore the impact of group membership; or they build models that attach 
group-level characteristics to individuals. All three approaches can result in inaccurate parameter 
estimates and the incorrect number of degrees of freedom; therefore, the results may lead to poor 
or even misleading policy analyses. Only recently have higher education researchers begun to 
recognize the need to analyze data taking into account the nested structures of institutions of 
higher education (Ethington, 1997; Patrick, 2001; Porter & Umbach, 2001). Multilevel modeling 
techniques allow researchers to appropriately handle the complex organizational effects of 
colleges and universities and provide the tools necessary to arrive at more accurate results. 

Using multilevel modeling to analyze survey data from over 1300 alumni from a large 
research university, this study proposes to examine the impact that departments have on student 
outcomes. Given that purpose, this study asks several questions: 

1 . What impact do individual characteristics such as race, gender, age, grade point average 
and transfer status have on satisfaction and the perceived impact that the college 
experience has on enhancing skill development? 

2. What effect do academic departments have on student satisfaction and the perceived 
impact that the college experience has on skill development? And to what extent do 
departmental size, departmental research focus, departmental diversity, and faculty 
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contact with students within a department relate to satisfaction and the perceived impact 
of college? 

Review of the literature 

Research suggests that alumni satisfaction is an excellent tool for assessing the effects of 
college on students (Pace, 1979; Pike, 1994). In an era where outcomes assessment is important, 
alumni surveys also play an important role in evaluating programs (Pike, 1994). While alumni 
gauging alumni satisfaction is important, little attention has been given to it since the 1970s and 
1980s (Bean & Bradley, 1986). 

Nevertheless, extensive research has been done on the various dimensions of college 
outcomes (Bowen, 1977; Feldman & Newcomb, 1969; Pascarella & Terenzini, 1991). In their 
review of the literature on student outcomes, Pascarella and Terenzini (1991) found that there are 
at least 10 dimensions of college outcomes. This study is informed in particular by research 
done Pascarella and Terenzini (1978) where they identified two particular dimensions of college 
outcomes - intellectual development and personal development. 

Individual effects 

While a great deal of literature focuses on using individual factors to predict student 
outcomes, very little work has been done on predicting student satisfaction. The literature 
suggests a complex relationship between grades and satisfaction with college. Some have 
suggested that grades a moderate relationship with satisfaction (Liu & Jung, 1980; Pike, 1991). 
Bean and Bradley (1986) found that grades had almost no relationship with student satisfaction. 
However, others have found that student performance (grade point average) significantly relates 
to satisfaction (Centra & Rock, 1983; Lavin, 1965); Likewise, given the models of student 
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retention where satisfaction and academic performance are predictors of attrition (Tinto, 1993), 
one could conclude that college grades and satisfaction are closely related. 

Several researchers have found significant gender differences in satisfaction with college 
(Adelman, 1991; Rienzi, Allen, Sarmiento, & McMillin, 1993). In most cases, women report 
lower satisfaction with college than do men. 

Scant research has been done on using race/ethnicity as a predictor of satisfaction. One 
study did find significant differences between racial/ethnic groups on reported satisfaction (Helm 
& Sedlacek, 1998). However, given the vast literature on the differences between races and the 
college experience, one would expect differences in satisfaction. Similarly, very little research 
has been done on the differences in satisfaction between transfer and native students. At the 
institution in this study, almost none of the transfer students live on campus, so one would expect 
their satisfaction to be similar to commuter students. Research suggests that transfer students are 
more critical of their college experience than native students (Liu & Jung, 1980). 

Departmental effects 

Beyond individual effects, research on the impact of college on students suggests that sub 
environments within the same institution can produce very different influences on students 
(Baird, 1988;Biglan, 1973b; Chickering, 1969; Weidman, 1989). Ewell (1989) argued that 
overall institutional culture was not significantly associated with student outcomes, but that 
major departments are important in the study of the impact of college on students. The literature 
points to departmental characteristics such as student-faculty relationships, structural diversity, 
and research emphasis of the faculty as factors influencing student outcomes. 
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Faculty continue to be one of the greatest influences on students’ experiences in college. 
A debate continues over the opposing roles of faculty work and how that impacts college 
students. Many have suggested that research and teaching are in conflict (Clark, 1987; Kerr, 
1963). Other researchers argue that research and teaching are complementary endeavors, and 
that faculty who do research are more likely to produce satisfied, well-educated students 
(Teague, 1981; Volkwein & Carbone, 1994). Still others (Feldman & Newcomb, 1969) suggest 
that there is no relationship between teaching and research. 

Whatever the relationship between research and teaching, faculty continue to be under 
attack. Many inside and outside of higher education have become increasingly concerned about 
how faculty spend their time (Bok, 1992; Boyer, 1990). Some have argued that faculty are now 
spending their discretionary time on research and are neglecting undergraduates (Massy & 
Zemsky, 1994). They suggest that as scholars “ratchet” up their research, student contact 
decreases, class size increases, and the number of faculty not engaging in undergraduate teaching 
decreases. Therefore, academic departments grow in size but not in service to their students. 

Peer influences also appear to relate to student outcomes. Milem (1998) concluded that 
peers significantly shape students attitudes and beliefs. What characteristics of peers within a 
department might shape students satisfaction? Perhaps departments with students with high 
academic ability shape student attitudes differently than those with students with low academic 
ability. 

Taking peer influences a step further, many researchers have investigated structural 
components of colleges and universities that impact students, particularly students of color and 
women (Gurin, 1999; Hurtado, Milem, Clayton-Pedersen, & Allen, 1999; Milem, in press; 

Milem & Hakuta, 2000). Chang (1996, 1998) found that students’ overall satisfaction with 
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college is enhanced in diverse learning environments. In other words, students have learning 
environments with a high proportion of students of color experience greater satisfaction than 
those who come from homogeneous learning environments. This is true for both white students 
and students of color. Although we are beginning to recognize the impact that diversity of 
colleges and universities has on student satisfaction, little empirical work has been done to 
support these ideas. 

Unit of analysis 

Given that both individual and departmental factors impact students, how does one best 
predict student outcomes? Some researchers (Berger, 2000; Berger & Milem, 2000) suggests a 
complex relationship between universities and students that cause methodological issues when 
determining units of analysis. Much of the current research focuses on how individual 
characteristics influence students. Several other studies chose to aggregate individual level 
outcomes data. Unfortunately analyses of this type are prone to what is known as an "ecological 
fallacy" (King, 1999; Kreft & DeLeeuw, 1998; Robinson, 1950), in which aggregate-level results 
may substantially differ or even be the reverse of individual-level results. To understand 
individual-level behavior, we must use individual-level data. 

In addition to aggregating individual-level data, researchers often attempt to address 
group level characteristics on a dependent variable by attaching group level variables to 
individual-level data. As Ethington (1997) notes, this solution is flawed for several reasons. 
First, it violates the fundamental assumption of OLS that observations are independent of one 
another. Second, it assumes that individuals within a group are affected identically by 
institutional characteristics. Finally, the attachment of group level variables to an individual 
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does not fully capture the effect of group level characteristics, which results in a misestimation of 
standard errors. 

We have long recognized that the impact of college is an interactive process between 
students and their environment (Astin, 1993; Weidman, 1989); yet prior to the recent availability 
of multilevel modeling techniques, all previous researchers have used simple cross-tabulations, 
correlations or regression analysis when studying college outcomes. Such techniques ignore 
these interactive processes the impact that institutions and their sub environments may have on 
outcomes. Clustering of the data can radically affect the substantive results of any analysis. Only 
by explicitly modeling this hierarchical structure of the data can we begin to truly understand 
why some how college impacts students. 

Sample and design 

This study uses data from a survey of alumnae at a large Research institution. The survey 
itself contained over eighty questions and was 4 pages long. The questions covered such topic 
areas as current employment status, satisfaction with various aspects of the institution, and self- 
assessed growth of skills and abilities. 

All 4,952 bachelor’s degree recipients for fiscal year 1999 were surveyed using the 
Dillman (1986, 2000) method of mail surveying in an effort to obtain high response rates. 
Dillman’s method involves multiple contacts with respondents when doing large-scale mail 
surveys, using a pre-notification contact and two survey mailings and reminder postcards. 

Taking into account bad addresses, the final sample size was 4,524. Of that sample, 1532 
surveys were returned yielding a response rate of 34 percent. 




10 



10 



Data analysis 

Based on the literature of student outcomes, we propose four multi-level models 
predicting student outcomes. We posit that satisfaction is related to individual attributes such as 
age, race, gender, transfer status and academic performance. In addition, we argue that 
departmental-level attributes such as research emphasis, size, student contact with faculty, 
academic ability of students, and diversity are related to satisfaction. 

Independent variables 

At the individual level (or in multilevel modeling terms, level- 1), we included the 
alumnus’ race/ethnicity, gender, age, cumulative grade point average (GPA), and transfer status. 
Means and standard deviations for all variables can be seen in Table 1. Race/ethnicity was 
included in the model using three dummy variables for African American, Asian Pacific 
American, and other students of color, with White as the reference category. We included 
gender and transfer status in the model as a dummy-coded variables as well. Grade point 
average at graduation and age at the time of the survey were two continuous variables included 
in the model at level- 1. Together these variables measure individual attributes of each alumnus 
that should affect their satisfaction and perceived gains in skills and abilities. 

We also included a second set of variables to ascertain the impact of departmental 
organization on student outcomes. These variables are all measured at the departmental level 
using department data collected in Fall 1997, approximately the junior or senior year of the 
individuals in our population. Henceforth we refer to these as group level, or level-2, variables. 

Three variables describe the undergraduate student body in each department. The 
proportion of female undergraduates and proportion of undergraduates of color are two 
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diversity measures. The midpoint of the 25 th and 75 th percentiles of the SAT for freshmen majors 
is used as a proxy for academic ability of students within a department. Because of the high 
correlation between SAT scores and college GPAs, we have not included SAT scores at the 
individual level. 

Three additional variables proxy the alumnus’ experience with faculty within a 
department. Instructional FTE is used to represent department size and is defined as the number 
of full-time equivalent instructional tenured or tenure-track faculty. We used average class size 
as a measure of student contact with faculty. It is the average size of lecture courses taught by a 
department not including specialized courses such as individual instruction or thesis courses. 
Labs and discussion sections are not included in this average. We calculate g rant dollars per 
FTE by dividing the total expenditures for research contracts and grants and dividing by the 
instructional FTE. Because of the non-normal distribution of grant dollars, we logged grant 
dollars per FTE to ensure normality (Tufte, 1 974). 

With such closely related variables in our level-2 model multicollinearity could be a 
major concern. An examination of the correlations shown in Table 2 reveals low correlations 
between the variables. These relatively low correlations confirm our assumption that the six 
group level variables included in our models measure different constructs. 

Dependent variables 

Before developing our models, we performed an exploratory factor analysis on questions 
related to general satisfaction and perceptions of the impact that the institution had in enhancing 
skills. The factor analyses yielded four factor structures similar to those found by Pascarella and 
Terenzini (1978). All four are used as dependent variables in our models: 

• Satisfaction with the major 
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• Personal skill development 

• Intellectual skill development 

• General skill development. 

Items comprising the four constructs, their loadings and reliability estimates are presented in 
Table 3. 

For satisfaction we developed one scale, satisfaction with the major (alpha=,72), which 
includes two Likert scale items. These asked alumni to rank the degree to which they agreed that 
their major provided them with a solid background for their career, and the degree to which they 
were satisfied with their major. 

We used a battery of 13 items that asked alumni to respond on a 3-point scale about the 
degree to which they believed their college experiences enhanced their skills to develop the other 
three constructs. The personal skill development scale (alpha=.76) included seven items that 
asked the extent to which college enhanced skills such as teamwork, leading others, professional 
ethics and understanding diverse cultural, political and intellectual views. Intellectual skill 
development (alpha=. 71) is a 7-item scale that asked the extent to which their college 
experiences enhanced skill such as writing, solving problems, thinking critically, and processing 
and interpreting data. The final scale, general skill development (alpha=.82), included 10 items 
from the other skill development scales such as writing effectively, speaking effectively, 
clarifying values, solving problems, and thinking creatively. 

Respondents were deleted from the final sample if they answered less than half of the 
questions related to development, if they answered none of the major satisfaction questions, or if 
their major was not located in one of the departments (e g. general studies). Mean substitutions 
were performed for the remaining missing variables prior to building the constructs. The 
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remaining 1,356 individuals from 54 departments were used as the final sample. The average 
department size was 24 respondents, ranging from one to 232. See Table 4 for department 
frequencies. 

Modeling strategy 

The first step in multilevel modeling is the calculation of the amount of variance in the 
dependent variable explained by group (department) membership, or the inter-class correlation 
(ICC). The ICC is calculated by building a fully unconditional model (often referred to as a one- 
way ANOVA) model) where no predictors are specified. The ICCs for the four dependent 
variables ranged from .06 (satisfaction with major and general development) to .08 (personal 
development. In other words, 6 to 8 percent of the variance in each of our dependent measures is 
explained by group membership. While these ICCs are somewhat modest, they are large enough 
to suggest that group membership has an impact on our outcomes and that multilevel modeling is 
the appropriate statistical approach. 

After calculating the ICCs, we added independent variables to the model. When adding 
independent variables we must consider how to center the variable and whether to enter it as 
random or fixed. Variables can have a fixed effect or a random effect. If theory guides one to 
assume that the impact of the independent variable will be the same for individuals in every 
group, the variable is said to have a fixed effect. If theory guides one to assume that the impact 
of the independent variable will be different for different groups, the variable is said to have a 
random effect, or is free to vary across groups. For variables that are said to have a random 
effect, their variance is partitioned into what is attributed to the group and what is attributed to 
the individual. However, because multilevel model requires massive amounts of data, the 
decision to include a random effect is often predicated by the number of groups in a data set and 
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the number of observations within each group. For our models, theory would suggest that 
perhaps race and gender be entered in our models as random effects, but due to the limitations in 
our data set, these models would not converge. 

As a result, we left only the intercept as random and could only model average group 
differences. Thus our models assume that the impact of the individual- and group-level variables 
is the same across departments; however, the average level of satisfaction within a department 
can vary across departments. For example, students in Department A could be much less 
satisfied than students in Department B, but if the average class size was reduced by the same 
amount in each department, satisfaction would increase by the same amount in each department. 

Centering is generally used to facilitate interpretation of the intercepts in the model. 
Variables are centered when the mean value of a variable is subtracted from the value the 
variable takes for each observation. If the mean is calculated across all observations, then the 
variable is grand mean centered; if the mean is calculated across observations in each cluster, 
then the variable is group mean centered. A general convention used in multilevel modeling is to 
grand mean center all fixed level- 1 variables and group mean center all random level- 1 variables. 
At level-2, in most cases, all independent variables are grand mean centered. So for all of our 
models, we have grand mean centered all level- 1 and level-2 variables. 

Results 

We estimated parallel models for each of our four independent variables. The full multi- 
level models can be seen in Table 5 with their variance estimates shown in Table 6. 

Satisfaction with education in the major 
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At the individual level, cumulative grade point average appears to be positively related 
with satisfaction with the major. Those with higher GPAs are more satisfied with their major. 
Gender and race also appear to have a significant relationship with satisfaction. Women and 
Asian Pacific Americans were on average less satisfied with their education in their major. 

At the group level, several contextual factors impact satisfaction with education in the 
major. The proportion of female undergraduates in a department is positively related to major 
satisfaction. Controlling for the other variables in the model, alumni in departments made up of 
a large proportion of women had greater satisfaction with the education in their major. 

Two other group-level variables appear to be significantly related with satisfaction with 
education in the major. The larger a department (as proxied by instructional FTE), the more 
satisfied on average students were with their education in their major. Focus on research also 
appears to be positively related to satisfaction with the major. As grant dollars per instructional 
FTE increases, the satisfaction that alumni had with their major increases. 

Personal skill development 

The model predicting alumni assessments of the degree to which their college experience 
enhanced their personal skill development (model 2) revealed several individual and group 
effects. At the individual level, only transfer status had a significant negative relationship with 
personal skill development. In other words, transfer students believed that their college 
experience enhanced their personal skill development significantly less than native students. 

At level-2, several group characteristics significantly predicted the mean score for 
personal development. Similar to satisfaction, the proportion of females had a significant 
positive relationship with assessment of the impact that college had on personal skill 
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development. The higher the proportion of females within a discipline, the higher the rating of 
the impact of college experiences on personal skill development. 

Likewise, both departmental size and departmental research emphasis were positively 
related to perceived college impact on personal skill development. The higher the instructional 
FTE in a department and the higher the grant dollars per FTE, the higher alumni rated the impact 
of college. 

Student contact also had a significant relationship with the dependent variable. As 
average class size increased, the less they believed that college enhanced their personal skill 
development. 

Intellectual skill development 

The model predicting intellectual skill development (model 3) appears somewhat 
different than that of personal skill development. At the individual level, gender, race and 
transfer status all had a significant relationship with the dependent variable. Females assessed 
the impact of college on intellectual development significantly lower than males. Likewise, 
transfer students assessed the impact significantly lower than native students. However, African 
American students rated the impact of college on intellectual development higher than White 
students. 

At the group-level, only one variable was found to be a significant predictor of the 
average perceived impact that college had on intellectual skill development. Departmental 
research focus, as proxied by grant dollars per FTE, was positively related with the dependant 
variable. 
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General skill development 

As expected, the fourth model, which predicts overall skill development, has some 
similarities to models 2 and 3. At the individual level race and transfer status are statistically 
significant predictors of the dependent variable. African Americans rate the impact that college 
had on enhancing their skills higher than Whites; Asian Pacific Americans rated the impact of 
college lower than Whites. Again, transfer students on average rated the impact of college 
significantly lower than native students. 

The model at level-2 would suggest a significant relationship between departmental size, 
student contact with faculty in the department, and gender make-up of the students in the 
department have a significant relationship with the average perceived impact of college on skill 
development. As the proportion of female undergraduates in a department increases, the 
perceived impact of college increases. When contact with faculty decreased (or average class 
size increased), average student ratings decreased; yet, as department size increased, average 
student ratings of the impact of college on skill development increased. 

Limitations 

While this study uses a powerful modeling technique, it is not without its limitations. 

First, we were limited to the number of variables we could use to adequately represent the 
culture of a department. Because of the size of the institution, a limited number of variables are 
collected and aggregated at the departmental level. We included the independent measures that 
we felt could best represent factors, as indicated in the literature, which may impact satisfaction. 

Second, we were limited by the data collected on the alumni survey. Many of the survey 
items were either mandated by the state coordinating body or were part of a previous survey that 
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could not be changed because of the administration’s desire to have longitudinal data. Although 
we were guided by the literature in the building of our scales, many of them could have been 
better measured by questions used on surveys that had been tested more extensively. 

Third, we had limited variability on many of the questions in the survey. Most of the 
survey responses were quite favorable, which may suggest some response bias. Whatever the 
reason, we were left with measures that had little variability. 

Finally, multilevel model requires a very large sample size, which we did not have. 
Because tests of significance are run at the group level, multilevel modeling requires a large 
number of groups with many individuals within each group. Ideally, we would have well over 
fifty groups with at least ten individuals per group. For this study, just over fifty groups were 
analyzed with several of the groups having less than ten members. As a result, we were only 
able to model the random intercept and no other random slopes even though theory would 
suggest that we should do otherwise. We also offer a word of caution about the generalizability 
of the study given our sample. The data used in the analysis were collected at only one 
institution, and any conclusions drawn from the results should be treated as such. 

Discussion 

Several important observations can be made about the analyses. First, the analyses 
provide a great deal of information about individual predictors of satisfaction. The relationship 
between GPA and our dependent measures appears to reflect the contradiction in the literature. 
GPA is a significant predictor of satisfaction with education in the major. Yet, there appears to 
be no relationship between GPA and the perceived impact of college on skill development. 
Furthering the complexity of the relationship between GPA and satisfaction is that some of the 
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literature suggests that the two may have a recursive relationship (Bean & Bradley, 1986; Pike, 
1991). 

Student race and gender also appear to be salient predictors of the college outcomes 
measured in this study. As with previous studies (Adelman, 1991; Rienzi et al., 1993), females 
consistently rated satisfaction and the college impact on development lower than males. The 
findings of the impact of race on the outcomes parallel previous research as well. (Helm & 
Sedlacek, 1998). African Americans in the sample reported higher ratings of the impact of 
college than did Whites. Asians consistently reported lower satisfaction with the education in 
their major and the impact of college on general skill development. 

These data suggest that transfer students have a different experience than native students. 
On all of the skill development scales, transfers rated the impact of college lower than native 
students. Few, if any, transfer students live on campus at any time during their academic career 
on campus. As a result, many do not become fully integrated into the campus and do not 
experience many of the co-curricular activities that have proven to impact students. In addition, 
transfer students spend fewer semesters at the institution compared with first-time students. 

Most interesting are the relationships between department characteristics and satisfaction 
and impact of college ratings. Racial diversity of departments appears to have little relationship 
with the outcomes in this study. This may be a true non-finding, but it may be attributed to the 
non-normality of the proportion of undergraduates measure. We cannot say conclusively that 
diversity does not matter; however, we cannot provide evidence that it does. 

Gender diversity of departments does appear to make a difference in impacting the 
outcomes we studied. The impact of the proportion of female undergraduates appears important, 
particularly in terms of satisfaction with the major and personal skill development. The data 




20 



20 



suggest that departments largely made up of female undergraduates have an impact on the 
college experience. 

This provides further support that the role of faculty appears to be extremely important in 
predicting student outcomes. In departments where faculty focus on research (as proxied by 
research dollars per FTE faculty) the students appear more satisfied with their major and report a 
higher impact of college experiences on skill development. Perhaps this suggests that 
departments where faculty are likely to get more research money, such as the sciences and 
engineering, that students are more satisfied. The prestige that comes with earning large grants 
may also have an impact on satisfaction. Nevertheless, these data may provide some support for 
the research that indicates that research and teaching are complementary rather than opposing 
activities. 

The size of a department, or number of faculty within a department, is important. 

Students from larger departments report higher satisfaction and a greater impact of college on 
skill development. While the number of faculty is important, contact with faculty is necessary as 
well. Students in departments with smaller average class size reported that college had a greater 
impact on their skill development than those from departments with larger classes. 

This evidence provides mixed support for the notion of the “academic ratchet” (Massy & 
Zemsky, 1994). The impact of class size does provide support for the ratchet, but the impact of 
research emphasis and department size seems to have an opposite effect than what the ratchet 
would suggest. 

The findings from our multilevel analysis of alumni survey data inform policy and 
practice in two areas. First, we argue that individual attributes matter. Students of different 
races/ethnicities and genders experience higher education differently and therefore have different 
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levels of satisfaction. One of the most apparent differences in satisfaction and perceptions of 
skill development is between transfer and native students. To increase overall satisfaction, 
institutions need to look at experiences of groups such as transfer students. 

Second, this study provides evidence of impact of organizational attributes on student 
outcomes. As focus on student satisfaction and first year-experience continues to grow, we need 
to go beyond comparing differences between institutions and begin to look at differences within 
institutions. In particular, we need to examine how departments are structured. Student contact 
with faculty, department size, gender diversity of departments, and departmental research 
emphasis all appear to have a significant relationship with satisfaction. 
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Table 1. Variable Means and Standard Deviations 



Variables 


Mean 


SD 


Dependent variables 


Satisfaction-major 


7.797 


1.683 


Intellectual development 


18.707 


2.676 


Personal development 


22.306 


2.639 


General development 


31.899 


3.936 


Individual-level independent variables 


GPA 


3.127 


0.496 


Age 


25.314 


4.369 


Female 


0.588 


0.492 


African American 


0.114 


0.318 


Asian 


0.134 


0.341 


Other Minority 


0.098 


0.297 


Transfer 


0.379 


0.485 


Group-level independent variables 


Prop. Nonwhite ugs 


0.271 


0.162 


Prop. Female Ugs 


0.561 


0.270 


SAT midpoint 


1,196.864 


72.361 


Average class size 


38.295 


18.880 


Instructional FTE 


19.049 


17.348 


Logged grants per instructional FTE 


9.104 


3.886 
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Table 2. Correlations of Level-2 Variables 



Logged 





Instructional Average 


Prop. 

Nonwhite 


Prop. 

Female 


SAT 


grants per 
Instructional 




FTE 


class size 


Ugs 


Ugs 


midpoint 


FTE 


Instructional FTE 


- 












Mean class size 


0.270 


- 










Proportion ugs nonwhite 


0.171 


0.258 


- 








Proportion ugs female 


- 0.372 


-0.064 


-0.186 


- 






SAT midpoint 


0.002 


-0.161 


0.074 


-0.137 


- 




Logged grants per instructional FTE 


0.228 


0.383 


0.232 


-0.258 


-0.168 


- 
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Table 3. Dependent Variable Scales 



Scales ] 


Factor loadings 


Satisfaction with major* 




Major provided a solid background 


0.89 


Satisfied with the quality of education in major 


0.89 


Alpha=. 73 




Enhanced personal development** 




Speaking effectively 


0.65 


Clarifying values 


0.63 


Teamwork 


0.69 


Leading others 


0.75 


Professional ethics 


0.70 


Understanding diverse cultural, pol., intell. views 


0.61 


Alpha=76 




Enhanced intellectual development** 




Writing effectively 


.47 


Solving problems 


.77 


Thinking creatively 


.70 


Thinking critically 


.77 


Computer/IT 


.47 


Processing/interpreting data 


.55 


Science and experimentation 


.51 


Alpha=.71 




Enhanced general development** 




Writing effectively 


0.52 


Speaking effectively 


0.62 


Clarifying values 


0.60 


Solving problems 


0.63 


Thinking creatively 


0.70 


Thinking critically 


0.69 


Teamwork 


0.61 


Leading others 


0.65 


Professional ethics 


0.64 


Understanding diverse cultural, pol., intell. views 


0.56 


Alpha=.82 





*l=strongly disagree, 2=diagree, 3=neutral, 4=agree, 5=strongly agree 
**l=not at all, 2=moderately, 3=extremely 
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Table 4. Departmental Frequencies 



Department 

AFRO/AMER ST 
AGR&RES ECON 
AMER STUDIES 
ANIMAL SCI 
ANTHROPOLOGY 
ARCH (B.S.) 

ART 

ART HISTORY 
ASIAN EAST EUR 
ASTRONOMY 
BIO RES ENGR 
BIO SCI 

BIOCHEMISTRY 

BUSINESS 

CLASSICS 

COMM 

COMPUTER SCI 

CRIM JUSTICE 

DANCE 

ECONOMICS 

ED-CURRIC & INSTR 

ED-HUMAN DEV 

SPECIAL ED 

ENGLISH 

ENGR-AEROSP 

ENGR-CHEM 

ENGR-CIVIL 

ENGR-ELEC 

ENGR-FIRE 

ENGR-MECH 

FAMLY STUDIE 

FOOD SCIENCE 

FRNITAL 

GEOGRAPHY 

GERMAN 

GOVT & POLIT 

HEALTH EDUC 

HEAR+SPCH SC 

HISTORY 

JEWISH STUDY 



N % of overall 



4 


0.3% 


1 


0.1% 


5 


0.4% 


14 


1 .0% 


6 


0.4% 


9 


0.7% 


21 


1.5% 


12 


0.9% 


8 


0.6% 


1 


0.1% 


12 


0.9% 


116 


8.6% 


12 


0.9% 


232 


17.1% 


1 


0.1% 


29 


2.1% 


45 


3.3% 


90 


6.6% 


5 


0.4% 


23 


1.7% 


72 


5.3% 


18 


1.3% 


7 


0.5% 


66 


4.9% 


10 


0.7% 


16 


1.2% 


22 


1.6% 


40 


2.9% 


11 


0.8% 


23 


1.7% 


35 


2.6% 


11 


0.8% 


4 


0.3% 


12 


0.9% 


3 


0.2% 


72 


5.3% 


23 


1.7% 


14 


1.0% 


24 


1.8% 


1 


0.1% 
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JOURNALISM 


35 


2.6% 


KINESIOL SCI 


45 


3.3% 


LINGUISTICS 


2 


0.1% 


MATHEMATICS 


6 


0.4% 


MICROBIOLOGY 


9 


0.7% 


MUSIC 


5 


0.4% 


NAT RES SCI 


10 


0.7% 


PHILOSOPHY 


2 


0.1% 


PHYSICS 


2 


0.1% 


PSYCHOLOGY 


53 


3.9% 


SOCIOLOGY 


32 


2.4% 


SPAN PORT 


14 


1.0% 


THEATRE 


7 


0.5% 


WOMENS STDYS 


4 


0.3% 


Total N 


1,356 




N of departments 


54 




Average department size 


25 
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Table 5. Full HLM Model Estimates and Standard Errors 
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Table 6. Full HLM Model Chi Square Table 







Parameter 






Random Effects 




Estimate 


DF 


Chi-square 


Model 1 


Intercept 


0.133 


47 


91.676** 




Residual 


2.670 






Model 2 


Intercept 


0.431 


47 


118.521 ** 




Residual 


6.513 






Model 3 


Intercept 


0.404 


47 


110.876** 




Residual 


6.502 






Model 4 


Intercept 


0.746 


47 


103.934** 




Residual 


14.499 







**p<.01, *p<.05, +p<. 10 



o 

ERIC 



34 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




NOTTCF. 



Reproduction Basis 




This document is covered by a signed "Reproduction Release 
(Blanket)" form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a "Specific Document" Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either "Specific Document" or "Blanket"). 




EFF-089 (3/2000) 





